Lesion area extraction apparatus, method, and program

ABSTRACT

Recording a plurality of lesion area extraction processing data generated in advance according to a plurality of types of lesion areas, recording a radiology which includes a character string having a lesion description character and being related to position information of a lesion area in the medical image, determining lesion area extraction processing data used for the extraction from the plurality of lesion area extraction processing data based on the lesion description character provided in the radiology report, and performing the extraction using the determined lesion area extraction processing data and the position information of the lesion area related to the character string.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a lesion area extraction apparatus andmethod using a radiology report of a medical image.

The invention also relates to a computer readable recording medium onwhich is recorded a program for causing a computer to perform themethod.

2. Description of the Related Art

In medical image diagnosis, it is often the case that image reading isperformed by a radiologist and the result of the reading is recorded andreported to the attending doctor as a radiology report, in whichelectronic radiology reports are widely used.

As one of such radiology reports, a report in which a lesion descriptioncharacter is linked to a reference image that includes the lesion areais known. Use of such radiology report allows the attending doctor toclick on the lesion description character in the report displayed on ascreen to display and observe a corresponding reference image, therebyfacilitating the understanding of the radiology report by the attendingdoctor.

In the image diagnosis, it is also practiced to evaluate the malignancyof a lesion or to decide the treatment policy based on the size orprogress of the lesion in the medical image. For such medical diagnosis,it is necessary to accurately extract a lesion area from a medicalimage. One lesion area extraction method is known as described, forexample, in U.S. Patent Application Publication No. 20080044080 in whichan arbitrary point is specified by the user in a lesion area of amedical image, a discrimination area is set with reference to thespecified point, an evaluation is made as to whether or not each pixelin the discrimination area is a pixel representing the contour of thelesion area, and the contour of the lesion area is determined based onthe evaluation result. Here, the evaluation as to whether or not eachpixel in the discrimination area is a pixel representing the contour ofthe lesion area is made by an evaluation function obtained throughmachine learning of multiple sample images, each including a lesion areawith a known contour.

In the mean time, extraction of a contour of a lesion area is conducted,in U.S. Patent Application Publication No. 20080044080, using the sameevaluation function regardless of the type of lesion.

Lesion areas, however, differ in characteristics from type to type andthe method that extracts all types of lesion areas by the sameevaluation function described above does not fully make use of thecharacteristics of the respective lesion areas, leading to thedegradation in extraction performance of a lesion area.

In view of the circumstances described above, it is an object of thepresent invention to provide a lesion area extraction apparatus andmethod capable of improving the extraction performance of a lesion area.It is a further object of the present invention to provide a computerreadable recording medium on which is recorded a program for causing acomputer to perform the method.

SUMMARY OF THE INVENTION

A lesion area extraction apparatus of the present invention is anapparatus for extracting a lesion area from a medical image, theapparatus, including:

an extraction process recording means for recording a plurality oflesion area extraction processing data generated in advance according toa plurality of types of lesion areas;

a report recording means for recording a radiology report which includesa character string having a lesion description character and beingrelated to position information of a lesion area in the medical image;

an extraction process determination means for determining lesion areaextraction processing data used for the extraction from the plurality oflesion area extraction processing data recorded on the extractionprocess recording means based on the lesion description characterprovided in the radiology report; and

an area extraction means for performing the extraction using thedetermined lesion area extraction processing data and the positioninformation of the lesion area related to the character string.

The term “lesion area extraction processing data” as used herein refersto a wide range of information data required for extracting a lesionarea from a medical image. Lesion area extraction processing datagenerated for a specific type of lesion area is lesion area extractionprocessing data generated by giving particular consideration to thecharacteristics of the specific type of lesion areas and is capable ofextracting a lesion area of the specific type. An evaluation functionobtained by machine learning a plurality of sample images in advance,each including, for example, a lesion area of a type called chest noduleto evaluate whether or not each pixel of a medical image is a pixelrepresenting a contour of the chest nodule is one such example.

The apparatus described above may further include a display means fordisplaying the radiology report and a specification means for specifyingthe character string in the displayed radiology report, the extractionprocess determination means may be a means that determines lesion areaextraction processing data based on the lesion description character ofthe specified character string, and the area extraction means may be ameans that extracts the lesion area using the determined lesion areaextraction processing data and the position information of the lesionarea related to the specified character string.

Further, the report recording means may be a means that further recordsa second radiology report generated, at a different time, for the samepatient as the one of a first radiology report which is the radiologyreport, the character string may be a character string also related to asecond character string, having a lesion description character, providedin the second radiology report, and, when the character string isrelated to the second character string, the extraction processdetermination means may be a means that determines the lesion areaextraction processing data based on the lesion description character ofthe second character string.

Still further, the report recording means may be a means that furtherrecords a second radiology report generated, at a different time, forthe same patient as the one of a first radiology report which is theradiology report, and the apparatus may be an apparatus further includesan obtaining means for, when a character describing a lesion at aposition corresponding to the position of a lesion area is provided inthe second radiology report, obtaining lesion area extraction processingdata for the described lesion, a judgment means for judging whether ornot the lesion area extraction processing data obtained by the obtainingmeans corresponds to the lesion area extraction processing datadetermined by extraction process determination means, and a warningmeans for issuing a warning if the judgment made by the judgment meansis not corresponding.

A lesion area extraction method of the present invention is a method forextracting a lesion area from a medical image, the method including thesteps of:

recording, on a extraction process recording means, a plurality oflesion area extraction processing data generated in advance according toa plurality of types of lesion areas; recording, on a report recordingmeans, a radiology report which includes a character string having alesion description character and being related to position informationof a lesion area in the medical image;

determining lesion area extraction processing data used for extractingthe lesion area from the plurality of lesion area extraction processingdata recorded on the extraction process recording means based on thelesion description character provided in the radiology report; and

extracting the lesion area using the determined lesion area extractionprocessing data and the position information of the lesion area relatedto the character string.

The method describe above may further include the steps of displayingthe radiology report and specifying the character string in thedisplayed radiology report, the step of determining lesion areaextraction processing data may be a step in which the determination isperformed based on the lesion description character of the specifiedcharacter string, and the step of extracting the lesion area may be astep in which the lesion area is extracted using the determined lesionarea extraction processing data and the position information of thelesion area related to the specified character string.

Further, the method may further include the step of recording a secondradiology report generated, at a different time, for the same patient asthe one of a first radiology report which is the radiology report, and,when the character string is a character string related to a secondcharacter string, having a lesion description character, provided in thesecond radiology report, the step of determining lesion area extractionprocessing data may be a step in which the lesion area extractionprocessing data is determined based on the lesion description characterof the second character string.

Still further, the method may further includes the steps of recording asecond radiology report generated, at a different time, for the samepatient as the one of a first radiology report which is the radiologyreport, when a character describing a lesion at a position correspondingto the position of a lesion area is provided in the second radiologyreport, obtaining lesion area extraction processing data for thedescribed lesion, judging whether or not the obtained lesion areaextraction processing data corresponds to the determined lesion areaextraction processing data, and issuing a warning if the judgment madein the judgment step is not corresponding.

A computer readable recording medium of the present invention is amedium on which is recorded a lesion area extraction program of thepresent invention for causing a computer to function as the extractionprocess recording means, report recording means, extraction processdetermination means, and area extraction means of the lesion areaextraction apparatus described above.

The program described above may be a program that further causes thecomputer to function as the display means for displaying the radiologyreport and the specification means for specifying the character stringin the displayed radiology report of the lesion area extractionapparatus described above.

Further, the program described above may be a program that still furthercauses the computer to function as the obtaining means, judgment means,and warning means of the lesion area extraction apparatus describedabove.

The program is supplied to the user recorded on a recording medium, suchas CD-ROM or DVD or recorded in downloadable to a built-in disk of aserver computer or a network storage device.

In the apparatus, method, and program described above, the term “type oflesion area” refers to the type of lesion indicated by the lesion area,and the type includes, by way of example, brain tumor, chest nodule,liver tumor, liver cyst, and kidney cyst.

The term “position information of a lesion area” as used herein refersto information indicating an arbitrary point in the lesion area or thearea.

The term “character string specification” as used herein refers to thata character string is specified by a mouse, a keyboard, or another inputdevice.

The term “lesion description character” as used herein refers to acharacter describing the lesion which includes, for example, any of thelesion name, a key word or a symbol representing the lesion name, and anabbreviation thereof.

The term “corresponding position” as used herein refers to the identicalposition or an adjacent position thereof.

According to the lesion area extraction apparatus, method, and programof the present invention, when extracting a lesion area from a medicalimage, lesion area extraction processing data used for extracting thelesion area from a plurality of lesion area extraction processing datagenerated in advance based on a lesion description character provided ina radiology report with respect to the lesion area, and the lesion areais extracted suing the determined lesion area extraction processingdata. This allows each lesion area to be extracted by making full use ofcharacteristics of each lesion area, whereby area extraction accuracymay be improved.

Further, the lesion area is extracted using the position information ofthe lesion area related to a character string of a radiology report, sothat the user does not need to directly specify an arbitrary point inthe lesion area, whereby the work load of the user may be reduced.

Further, in the lesion area extraction apparatus, method, and programdescribed above, if a radiology report is displayed, then a characterstring is specified in the displayed radiology report, lesion areaextraction processing data is determined based on the lesion descriptioncharacter of the specified character string, and the lesion area isextracted using the determined lesion area extraction processing dataand the position information of the lesion area related to the specifiedcharacter string, the lesion area corresponding to the lesiondescription character of the character string may be extractedaccurately by making full use of the characteristics of the lesion area.Further, by displaying the extracted lesion area, radiology reportgeneration or observation efficiency may be improved.

Still further, where the character string is related to a secondcharacter string having a lesion description character provided in asecond radiology report generated, at a different time, for the samepatient as the one of a first radiology report which is the radiologyreport thereof, if lesion area extraction processing data is determinedbased on the lesion description character of the second characterstring, the lesion area may be extracted by the same lesion areaextraction processing data as that used for extracting the relatedlesion area observed at a different time. In particular, where characterstrings describing the same lesion are related to each other, the lesionarea may be extracted by the same lesion area extraction processing dataas that used for extracting an area of the same lesion observed at adifferent time.

Further, where a character describing a lesion at a positioncorresponding to the position of the lesion area is provided in a secondradiology report generated, at a different time, for the same patient asthe one of a first radiology report which is the radiology reportthereof, if an arrangement is adopted in which lesion area extractionprocessing data for the described lesion is obtained, then a judgment ismade whether or not the obtained lesion area extraction processing datacorresponds to the determined lesion area extraction processing data,and a warning is issued the judgment made is not corresponding, thenwhen the determined lesion area extraction processing data differs fromthe lesion area extraction processing data for a lesion which is highlylikely the same lesion observed at a different time, revision of thedetermined lesion area extraction processing data may be prompted byissuing a warning. This allows modification to be made in the determinedlesion area extraction processing data according to the user input inresponse to the warning.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram of an embodiment of thelesion area extraction apparatus of the present invention.

FIG. 2 shows an example screen for generating a radiology report.

FIG. 3 shows an example screen for generating a radiology report.

FIG. 4 shows an example screen for generating a radiology report.

FIG. 5 is a block diagram of the lesion area extraction apparatus.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings. FIG. 1 is a block diagram of amedical image processing system, schematically illustrating theconfiguration thereof. As shown in FIG. 1, the system includes modality10, data storage server 20, radiology report generation unit 30, andlesion area extraction unit 40 which are communicatably linked to eachother via network 9.

Modality 10 is a system for generating a medical image representing atest body, and more specifically it is a CT system, MRI system, PET,ultrasonic diagnostic system, or the like. The image generated bymodality 10 is sent to data storage server 20 and stored therein.

Data storage server 20 is a computer for storing/managing various typesof data and communicates with other components via network 9 tosend/receive image data and the like. More specifically, a medical imagegenerated by modality 10, a radiology report generated by radiologyreport generation unit 30, and the like are obtained through network 9,stored in a recording medium, such as a hard disk, and managed. Further,server 20 retrieves data in response to a retrieval request from theother system component, such as radiology report generation unit 30 orlesion area extraction unit 40, and sends the extracted data to thesystem component that has made the retrieval request.

Since data storage server 20 includes medial images and radiologyreports with tag information identifying respective patients attachedthereto, a medical image and a radiology report of the same patient canbe easily retrieved and extracted for use using the tag information.

The image data storage format and communication between each componentof the system via network 9 is based on DICOM protocol or the like.

Radiology report generation unit 30 displays, for example, radiologyreport generation screen 100 or 100 a shown in FIG. 2 or 4, andgenerates an electronic radiology report of a medical image according touser input operation in the generation screen. For that purpose, unit 30includes: a display, such as a liquid crystal display or the like, forperforming various type of displays; an input device of a keyboard, amouse, and the like for use by the user to input various instructions; ahard disk for storing various types of data including image data andradiology reports; a memory in which a radiology report generationprogram is stored; a CPU for controlling the radiology report generationunit by executing the program stored in the memory; and a communicationinterface linked to network 9 via a bus.

Radiology report generation unit 30 may generate a radiology report inwhich a character string is related by hyperlink to a medical image,position information of a lesion area of the medical image, or acharacter string of another radiology report. The attending doctor orthe like may click on the character string (link character) of theradiology report on the screen to display a medical image or the likerelated to the character string on the display.

In the present application, a link character not related to a characterstring of another radiology report is called a first class linkcharacter and that related to a character string of another radiologyreport is called a second class link character.

First, radiology report generation screen 100 and processing forproviding a first class link character in a radiology report accordingto user input operation will be described.

Radiology report generation screen 100 includes patient information area101 for providing patient information, medical image information area103 for providing medical image information, radiology report area 104,and the like. With respect to radiologist name field 104A, finding field105 which is a radiology report, and diagnostic field 107 provided inradiology report area 104, information may be inputted and edited by adoctor or the like as the operator using a mouse, a keyboard, or thelike. In addition, radiology report generation screen 100 includes linkinput selection button 111 that allows a link character linked byhyperlink to a medical image to be inputted in finding field 105.

Radiology report generation screen 100 further includes radiology reportediting button 109 for storing or canceling an edited radiology report,past report selection button 119A for referencing various types ofinformation, and reference image selection button 119B. For example, ifreference image selection button 119B is operated, thumbnails of medicalimages are selectably displayed in reference information area 117, asshown in FIG. 2, and when one of the thumbnails is further selected,medical image 115 corresponding to selected thumbnail 117A is displayedin detailed image display area 113. Detailed image display area 113 isprovided with editing buttons 113A and 113B for processing and editingthe medical image.

Next, processing performed in radiology report generation unit 30 forproviding a link character 105A corresponding to a first class linkcharacter in finding field 105 will be described. First, link inputbutton 111 is selected to set the link input mode to ON. Then, when across pointer is placed on the position of a lesion and mouse clicked,the link character 105A linked to medical image 115, including theposition 115A of the lesion, is related to position information thatindicates the position of the lesion area, such as the coordinatesthereof, in the medical image 115 and inserted in finding field 105.Here, the link character 105A is inserted in finding field 105 in amanner so as to be distinguishable from other characters and a secondclass link character, such as coloring, underlining, or the like, inorder to indicate that the link character 105A is linked.

In radiology report generation unit 30, a fixed character string, suchas lesion 1 or lesion 2, is inserted as the link character 105A only byspecifying the position of a lesion area, so that the operator may editthe inserted link character and a text around the character or mayinsert a different link character through the use of a keyboard or thelike later. FIG. 3 shows an example of finding field 105 of radiologyreport generation screen 100. The example finding field of FIG. 3includes three different link characters lesion 1 (105A), lesion 2(105B), and lesion 3 (105C) and shows that lesion 1 (105A) is changed to“chest nodule (105A)” which are the characters of more specificallydescribing the lesion 1.

When a doctor generating or observing the radiology report specifies afirst class link character, a medical image linked to the link characteris displayed on the display and an indicator, such as a cross mark, anarrow, or the like, indicating the position of the lesion is displayedon the medical image.

Next, radiology report generation screen 100 a and processing forproviding a second class link character in a radiology report accordingto user input operation will be described.

Radiology report generation screen 100 a includes, on the left side,present image area 116 for displaying present image 116 a which is amedical image obtained through the present inspection and the target ofradiology reading and present report area 106 for displaying a presentreport which is a radiology report generated by performing radiologyreading of the present image. While, the screen 100 a includes, on theright side, past image area 118 for displaying past image 118 a which isa medical image obtained through the previous inspection with respect tothe same patient and previous report area 108 for displaying a previousreport which is a radiology report generated by performing radiologyreading on the previous image. With respect to present image area 116and present report area 106, information may be inputted and edited by adoctor or the like as the operator using a mouse, a keyboard, or thelike.

Next, processing performed in radiology report generation unit 30 forproviding a link character 106A corresponding to a second class linkcharacter in finding field 106 will be described. When a cross pointeris placed on the position of a lesion in a present image 116 a and theright button of a mouse is clicked, a list of selection items, such as“To Report”, “Gradation”, “Enlargement”, and the like is displayed inpull-down menu format, and if a selection item of “To Report” isselected, a list of character string candidates to be inserted infinding field 106 is displayed in pull-down menu format.

As the candidates to be inserted, the list displays as selection items,for example, lesion names, such as bulla, pneumothorax, emphysema,bronchodilation, and the like in addition to a link character and thelike provided in the previous report. FIG. 4 shows an example case inwhich a link character of nodule (108A) is provided in the previousreport and a selection item of “previous lesion (nodule)” representingthe link character is included in the list. Here, the description willbe made on the assumption that the link character “nodule” (108A) is afirst class link character related to position information of a lesionarea in the previous image.

The user moves the selection frame 116B upward or downward by keyboardor mouse operation to select a character string to be inputted tofinding field 106 and the selection is finalized. For example, if akeyboard is used, the selection is finalized by pressing the “Enter”key. It is noted that when selection frame 116B is on a selection itemof “previous lesion (nodule)”, the link character “nodule” (108A) in theprevious report corresponding to the selection item is highlighted.

As for the selection item representing the link character provided inthe previous report, all link characters provided in the previous reportmay be employed or only a link character provided in relation to theposition in the previous image corresponding to the position 116A in theright-mouse clicked present image may be employed. The correspondingpositions as used herein refer to the same or adjacent positions.

Next, when “previous lesion (nodule)” is selected from the displayedlist and the selection is finalized, a link character (106A) of “nodule”is inserted in the present report area 106 which is related to the linkcharacter “nodule” (108A) in the previous report corresponding to theselected item. Further, the link character “nodule” (106A) is related toposition information, such as the coordinates, of position 116A oflesion area in the present image 116 a on which the right button of themouse is clicked. The link character 106A is inserted in a manner so asto be distinguishable from other characters and a first class linkcharacter, such as coloring, underlining, or the like, in order toindicate that the link character is linked to a character string inanother radiology report.

When a doctor generating or observing the radiology report specifies asecond class link character of “nodule” (106A), a present image 106 alinked to the link character is displayed on the display and anindicator, such as a cross mark, an arrow, or the like, indicating theposition 116A of the lesion is displayed in the present image 106 a.Further, the previous image 118 a linked to the character string“nodule” (108A) in the previous report which is in turn linked to thelink character “nodule” (106A) is displayed on the display and anindicator, such as a cross mark, an arrow, or the like indicating theposition of the lesion is displayed in the previous image 118 a. As aresult, comparative observation of a temporal change in the nodule,which is a common lesion between the present and previous images, can bemade easily.

Here, the description has been made of a case in which the previousreport is made referable when generating a present report by performingradiology reading of the present image, but not limited to this, and aradiology report generated in the past for the same patient as that ofthe present report to be generated can be made referable for generatingthe present report.

When a plurality of inspections is performed for the same patient and aradiology report is generated with respect to each medical imageobtained in each inspection, radiology report generation unit 30provides a list of link characters, like that shown in Table 1, used ineach radiology report.

TABLE 1 LINK INSPECTION DATE AND TIME CHARACTER 2008.02.13 2008.01.222007.10.12 2007.07.13 NODULE ◯ ◯ ◯ ◯ PLEURAL ◯ ◯ EFFUSION PNEUMONIA ◯

In Table 1, “◯” mark represents a link character provided in eachradiology report. Further, by providing each “◯” mark with the same linkas that of each corresponding link character, the linked medical image,the position of a lesion, and the like may be displayed on the displaywhen either one of “◯” marks is specified by the user, like when thecorresponding link character is specified in a radiology reportdisplayed on the screen.

Further, each data of inspection (2008.02.13, or the like) in Table 1may be provided with links to corresponding medical image and radiologyreport to allow the user, when the user specifies any one of the datesof inspection, to display the medical image and radiology reportcorresponding to the date of inspection specified by the user. Further,a configuration may be adopted in which, when any one of link characters(nodule and the like) is specified, a list of medical images of thelesion corresponding to the link character is displayed to facilitateunderstanding of a temporal change in the lesion.

As a standard computer configuration, lesion area extraction unit 40includes CPU 2, memory 3, hard disk 4, and communication port 7, asillustrated in FIG. 5. Further, display 5 and an input device, such asmouse 6, are connected to lesion area extraction unit 40.

A lesion area extraction program is stored in memory 3. The lesion areaextraction program defines, as processes executed by CPU 2, anextraction processing data recording process, a report recordingprocess, a report display process, a link character specificationprocess, an extraction processing data determination process, anextraction processing data obtaining process, a judgment process, awarning process, an extraction processing data changing process, alesion area extraction process, a lesion area displaying process, andthe like. When these processes are executed by CPU 2 according to theprogram, a general purpose computer functions as a display means, aspecification means, an extraction process recording means, a reportrecording means, an extraction process determination means, an obtainingmeans, a judgment means, a warning means, a changing means, areaextraction means, and the like.

Hard disk 4 includes a plurality of lesion area extraction processingdata, each being generated according to each of a plurality of types oflesion areas, radiology reports obtained from data storage server 20 andthe like via network 9, and the like.

Specific processing performed by lesion area extraction unit 40 forextracting a lesion area from a medical image will now be described.

First, prior to initiating the lesion area extraction process, lesionarea extraction unit 40 records, on hard disk 4, a plurality of lesionarea extraction processing data, each being generated in advanceaccording to each of a plurality of types of lesion areas. Here, each ofthe plurality of lesion area extraction processing data is an evaluationfunction obtained with respect to each type of lesions, such as braintumor, chest nodule, liver tumor, liver cyst, kidney cyst, and the likethrough machine learning a plurality of sample images which include eachtype of lesion area in advance to evaluate whether or not each pixel ina medical image is a pixel representing a contour of each type of lesionarea. Each of the evaluation functions may be obtained through machinelearning based on, for example, the AdaBoosting algorithm described inU.S. Patent Application Publication No. 20080044080.

Then, lesion area extraction unit 40 obtains (records) and displays aradiology report in response to a request from the operator. Lesion areaextraction unit 40 obtains a particular radiology report correspondingto the operator's request from data storage server 20 or radiologyreport generation unit 30 via communication port 7, and records theobtained radiology report on hard disk 4, as well as displaying on thedisplay 5. The identification of the radiology report is based on inputof specifying patient name information, patient ID, inspection ID, orradiology report ID.

Then, in the particular radiology report (target report) displayed ondisplay 5, a specification process for specifying a link character isperformed. For example, in response to the selection, by the operator,of any of link characters provided in the finding filed of the targetreport with the input device such as mouse 6, the clicked link characteris specified. Here, it is assumed that one or more of either or both offirst and second class link characters are included in the radiologyreport and the link character specified by the operator is a first classlink character or a second class link character.

Next, based on the link character specified by the link characterspecification process, lesion area extraction unit 40 performs a processfor determining lesion area extraction processing data used forextracting the lesion area from a plurality of lesion area extractionprocessing data recorded on hard disk 4. The process of determining thelesion area extraction processing data when the link character specifiedin the link character specification process is a first class linkcharacter differs from the process when the link character specified inthe link character specification process is a second class linkcharacter. Therefore, each case will be described hereinafter.

First, the case in which the link character specified by the linkcharacter specification process is a first class link character will bedescribed. When a first class link character is specified by the linkcharacter specification process, lesion area extraction unit 40 performsa process of determining the lesion area extraction processing databased on a lesion description character in the specified link character.More specifically, with respect to each of a plurality of types oflesions, lesion description character that includes, for example, any ofthe lesion name, a key word or a symbol representing the lesion name,and an abbreviation thereof is related to one of a plurality of lesionarea extraction processing data recorded on hard disk 4 which seems tobe most appropriate for extracting a lesion area of the type indicatedby the lesion description character and registered in advance, and thelesion area extraction processing data corresponding to the lesiondescription character of the link character is selected from theregistered list of lesion description characters. In order to preventretrieval omission, it is preferable to also register a synonym(Japanese/English) of each registered lesion description character asbeing representing each registered lesion description character, therebyperforming the retrieval based on each registered lesion descriptioncharacter and the list of the synonyms.

Then, the lesion area extraction processing data retrieved as beingrelated to the lesion description character is determined to be thelesion area extraction processing data for extracting the lesion area.For example, lesion description characters of brain tumor, chest nodule,liver tumor, liver cyst, and kidney cyst are related to lesion areaextraction process function F1 obtained in advance through machinelearning of a plurality of sample images of brain tumor, lesion areaextraction process function F2 obtained in advance through machinelearning of a plurality of sample images of chest nodule, lesion areaextraction process function F3 obtained in advance through machinelearning of a plurality of sample images of liver tumor, and lesion areaextraction process function F4 obtained in advance through machinelearning of a plurality of sample images of liver cyst, and lesion areaextraction process function F5 obtained in advance through machinelearning of a plurality of sample images of kidney cyst respectively andregistered in advance. Then, in the link character specificationprocess, when, for example, the link character (105A) formed of acharacter string of “chest nodule” in FIG. 3 is specified, the lesionarea extraction process function F2 related to the lesion descriptioncharacter, that is, “chest nodule” is determined to be the lesion areaextraction process function for extracting the lesion area.

Where no matching lesion description character is found, a lesiondescription character having highest similarity to the character stringmay be retrieved. Alternatively, a list of lesion description charactercandidates may be displayed in pull-down menu format to have theoperator to select a lesion description character having highestassociation from the list and the lesion area extraction processing datarelated to the selected lesion description character is determined to bethe lesion area extraction processing data for extracting the lesionarea. Here, if a lesion description character is selected from the listof lesion description character candidates displayed in pull-down menuformat, the selection result is recorded and when the same linkcharacter is specified in the subsequent processing, the lesion areaextraction processing data to be used is determined based on therecorded selection result.

Next, the case in which the link character specified by the linkcharacter specification process is a second class link character will bedescribed. When a second class link character is specified by the linkcharacter specification process, lesion area extraction unit 40 performsa process of determining the lesion area extraction processing databased on a lesion description character in a character string in anotherradiology report related to the specified link character. For example,when a link character “previous lesion” related to a link character“nodule” in another radiology report of the same patient generated atdifferent time is found in the target report, and the link character“previous lesion” is specified in the link character specificationprocess, a region description character corresponding to the linkcharacter “nodule” in another radiology report related to the specifiedlink character “previous lesion” is retrieved from the list of lesiondescription characters registered on hard disk 4 in advance and lesionarea extraction processing data related to the corresponding lesiondescription character is determined to be the lesion area extractionprocessing data for extracting the lesion area.

That is, the second case differs from the first case where the firstclass link character is specified on which link character is used todetermine the lesion area extraction processing data. But, the processof determining a most appropriate extraction processing data from aplurality of lesion area extraction processing data registered on harddisk 4 in advance is common, and with respect to the common process,various technical aspects employed in the case where the first classlink character is specified may be applied directly.

Further, lesion area extraction unit 40 has a function to select a moredetailed lesion area extraction processing data based on the lesion areaextraction processing data determined based on each link character(first class link character or second class link character) in themanner as described above or to change a parameter of the lesion areaextraction processing data based on information tagged to a medicalimage (DICOM information, such as slice interval and the like) linked tothe link character.

Here, the term “select a more detailed lesion area extraction processingdata” as used herein refers to, if, for example, a first chest nodulelesion area extraction processing data (chest nodule 1) obtained bymachine learning sample images with a slice interval of 5 mm or greaterand a second chest nodule lesion area extraction processing data (chestnodule 2) obtained by machine learning sample images with a sliceinterval of less than 5 mm are provided as chest nodule lesion areaextraction processing data, that either chest nodule 1 (slice intervalof 5 mm or greater) or chest nodule 2 (slice interval of less than 5 mm)is selected.

The term “change a parameter” as used herein refers to set spacing or aslice interval, a slice thickness, or the like to the lesion areaextraction processing data as a parameter.

When a character describing a lesion located at a position correspondingto the position of the lesion area (position of target lesion) relatedto the specified link character is found in another radiology report ofthe same patient as that of the target report, lesion area extractionunit 40 performs a judgment process for judging whether or not theprocess of obtaining the lesion area extraction processing data for thedescribed lesion, the obtained lesion area extraction processing data,and the determined lesion area extraction processing data are matched,and a warning process for issuing a warning if they do not match in thejudgment.

More specifically, a judgment is made as to whether or not a lesion ispresent at a position corresponding to the position of the targetlesion. For example, when a previous image and a previous report, whichis a radiology report of the previous image, for the same patient asthat of the target report are stored in data storage server 20, all linkcharacters provided in the previous report are obtained and, based onposition information of lesion area in the previous image related toeach link character, a judgment is made as to which of the positions isidentical to the position of the target lesion or an adjacent positionthereof. Preferably, the judgment is made after aligning the previousimage with the medical image that includes the target lesion using animage alignment technology described, for example, in JapaneseUnexamined Patent Publication No. 2007-068554 to match the referenceposition in position coordinates between each image.

As a result of the judgment, if no lesion is found at a positioncorresponding to the position of the target lesion or an adjacentposition thereof, a judgment is made that no lesion is present at thecorresponding position, and subsequent obtaining process for obtaininglesion area extraction processing data, judgment process, and warningprocess are not performed. On the other hand, if a lesion is found at aposition corresponding to the position of the target lesion or anadjacent position thereof, a judgment is made that the lesion present atthe position corresponding to the position of the target lesion or anadjacent position thereof is the lesion present at the correspondingposition. Where two or more lesions are present at the positioncorresponding to the position of the target lesion or an adjacentposition thereof, a lesion located at a position closest to the positionof the target lesion is determined to be the lesion at the correspondingposition.

Next, the lesion area extraction processing data for the lesion judgedto be present at the corresponding position is obtained. If any oflesion area extraction processing data was applied to the lesion in thepast and the history thereof is recorded, the lesion area extractionprocessing data applied in the past is obtained. If no such history isrecorded, the lesion area extraction processing data determinationprocess described above is applied to the link character in the previousreport with respect to the lesion to obtain the most appropriateextraction processing data for extracting the area.

Then, a judgment is made as to whether or not the obtained lesion areaextraction processing data corresponds to the lesion area extractionprocessing data determined in the lesion area extraction processing datadetermination process. If the judgment result shows that they correspondto each other, the subsequent warning process is not performed. On theother hand, if they do not correspond to each other, a warning is issuedto notify the user of the disagreement. For example, a message promptingreview of the determined lesion area extraction processing data isdisplayed on display 5. In this case, it is preferable that the lesionarea extraction processing data obtained for the lesion at thecorresponding position is also displayed.

When an instruction to change the lesion area extraction processing datais inputted by the user in response to the outputted warning, lesionarea extraction unit 40 performs a process of changing the extractionprocessing data according to the instruction. For example, according tothe instruction from the user, the determined lesion area extractionprocessing data may be changed to lesion area extraction processing dataobtained for the lesion at the corresponding position or another lesionarea extraction processing data stored in hard disk 4. When the lesionarea extraction processing data is changed, the changed lesion areaextraction processing data is used for an area extraction process to bedescribed later. In the mean time, if a change instruction is notinputted by the user or an instruction to continue the processingwithout change is inputted, the subsequent processing is continuedwithout changing the lesion area extraction processing data.

The obtaining process, judgment process, warning process, and changingprocess may be performed only when the link character specified in thelink character specification process is a first class link character.

Then, an area extraction process is performed using the lesion areaextraction processing data determined in the extraction processing datadetermination process (if changed after the determination, then thechanged lesion area extraction processing data) and position informationof the lesion area related to the link character specified in the linkcharacter specification process. More specifically, with reference tothe position of the lesion area related to the link character, ajudgment area, including the lesion area, is set in the medical image.As for the judgment area, for example, an area greater than a maximumpossible size of the lesion area is set with the position of the lesionarea in the center. Then, a characteristic amount of each pixel in thejudgment area is obtained in order to evaluate whether or not each pixelis a pixel representing a contour of the lesion area. The characteristicamount is determined according to the lesion area extraction processingdata used for the evaluation, such as luminance information of anadjacent area of each pixel. Next, based on the obtained characteristicamount, an evaluation value that indicates whether or not each pixel inthe judgment area is a pixel representing a contour of the lesion areausing the lesion area extraction processing data determined in theextraction process determination process. Then, based on the evaluationvalue calculated for each pixel, the lesion area is extracted by dynamicprogramming or graph cut method described, for example, in U.S. PatentApplication Publication No. 20080044080.

Then, lesion area extraction unit 40 performs a process of displaying amedical image in which the extracted lesion area is presented in amanner distinguishable from other areas with a radiology report thereofon display 5.

According to the lesion area extraction unit, method, and program of thepresent embodiment, a doctor generating or observing a radiology reportmay easily refer to the size or progress of a lesion by specifying thelink character of the lesion provided in the finding field of theradiology report, so that the doctor may generate or observe theradiology report more efficiently. Further, each lesion area isextracted by fully utilizing the characteristics thereof, whereby regionextraction accuracy may be improved.

In the embodiment described above, the description has been made of acase in which, in response to the specification of a link character bythe operator, a lesion area corresponding to the specified linkcharacter is extracted. But an arrangement may be adopted in which, whena radiology report is obtained from data storage server 20 or the like,a process for extracting a lesion area corresponding to each of all linkcharacters included in the radiology report is performed and theextraction result of lesion areas are recorded on hard disk 4 or thelike to allow them to be outputted in various forms desired by theoperator at a later time.

1. A lesion area extraction apparatus for extracting a lesion area froma medical image, the apparatus comprising: an extraction processrecording means for recording a plurality of lesion area extractionprocessing data generated in advance according to a plurality of typesof lesion areas; a report recording means for recording a radiologyreport which includes a character string having a lesion descriptioncharacter and being related to position information of a lesion area inthe medical image; an extraction process determination means fordetermining lesion area extraction processing data used for theextraction from the plurality of lesion area extraction processing datarecorded on the extraction process recording means based on the lesiondescription character provided in the radiology report; and an areaextraction means for performing the extraction using the determinedlesion area extraction processing data and the position information ofthe lesion area related to the character string.
 2. The lesion areaextraction apparatus of claim 1, wherein: the apparatus furthercomprises a display means for displaying the radiology report and aspecification means for specifying the character string in the displayedradiology report; the extraction process determination means is a meansthat performs the determination based on the lesion descriptioncharacter of the specified character string; and the area extractionmeans is a means that performs the extraction using the determinedlesion area extraction processing data and the position information ofthe lesion area related to the specified character string.
 3. The lesionarea extraction apparatus of claim 1, wherein: the report recordingmeans is a means that further records a second radiology reportgenerated, at a different time, for the same patient as the one of afirst radiology report which is the radiology report; the characterstring is a character string also related to a second character string,having a lesion description character, provided in the second radiologyreport; and when the character string is related to the second characterstring, the extraction process determination means is a means thatdetermines the lesion area extraction processing data based on thelesion description character of the second character string.
 4. Thelesion area extraction apparatus of claim 1, wherein: the reportrecording means is a means that further records a second radiologyreport generated, at a different time, for the same patient as the oneof a first radiology report which is the radiology report; and theapparatus further comprises: an obtaining means for, when a characterdescribing a lesion at a position corresponding to the position of alesion area is provided in the second radiology report, obtaining lesionarea extraction processing data for the described lesion; a judgmentmeans for judging whether or not the lesion area extraction processingdata obtained by the obtaining means corresponds to the lesion areaextraction processing data determined by extraction processdetermination means; and a warning means for issuing a warning if thejudgment made by the judgment means is not corresponding.
 5. The lesionarea extraction apparatus of claim 2, wherein: the report recordingmeans is a means that further records a second radiology reportgenerated, at a different time, for the same patient as the one of afirst radiology report which is the radiology report; the characterstring is a character string also related to a second character string,having a lesion description character, provided in the second radiologyreport; and when the character string is related to the second characterstring, the extraction process determination means is a means thatdetermines the lesion area extraction processing data based on thelesion description character of the second character string.
 6. Thelesion area extraction apparatus of claim 2, wherein: the reportrecording means is a means that further records a second radiologyreport generated, at a different time, for the same patient as the oneof a first radiology report which is the radiology report; and theapparatus further comprises: an obtaining means for, when a characterdescribing a lesion at a position corresponding to the position of alesion area is provided in the second radiology report, obtaining lesionarea extraction processing data for the described lesion; a judgmentmeans for judging whether or not the lesion area extraction processingdata obtained by the obtaining means corresponds to the lesion areaextraction processing data determined by extraction processdetermination means; and a warning means for issuing a warning if thejudgment made by the judgment means is not corresponding.
 7. A lesionarea extraction method for extracting a lesion area from a medicalimage, the method comprising the steps of: recording, on a extractionprocess recording means, a plurality of lesion area extractionprocessing data generated in advance according to a plurality of typesof lesion areas; recording, on a report recording means, a radiologyreport which includes a character string having a lesion descriptioncharacter and being related to position information of a lesion area inthe medical image; determining lesion area extraction processing dataused for the extraction from the plurality of lesion area extractionprocessing data recorded on the extraction process recording means basedon the lesion description character provided in the radiology report;and performing the extraction using the determined lesion areaextraction processing data and the position information of the lesionarea related to the character string.
 8. The lesion area extractionmethod of claim 7, wherein: the method further comprising the steps ofdisplaying the radiology report and specifying the character string inthe displayed radiology report; the step of determining lesion areaextraction processing data is a step in which the determination isperformed based on the lesion description character of the specifiedcharacter string; and the step of performing the extraction is a step inwhich the extraction is performed using the determined lesion areaextraction processing data and the position information of the lesionarea related to the specified character string.
 9. The lesion areaextraction method of claim 7, wherein: the method further comprises thestep of recording a second radiology report generated, at a differenttime, for the same patient as the one of a first radiology report whichis the radiology report; and when the character string is a characterstring related to a second character string, having a lesion descriptioncharacter, provided in the second radiology report, the step ofdetermining lesion area extraction processing data is a step in whichthe lesion area extraction processing data is determined based on thelesion description character of the second character string.
 10. Thelesion area extraction method of claim 7, further comprising the stepsof: recording a second radiology report generated, at a different time,for the same patient as the one of a first radiology report which is theradiology report; when a character describing a lesion at a positioncorresponding to the position of a lesion area is provided in the secondradiology report, obtaining lesion area extraction processing data forthe described lesion; judging whether or not the obtained lesion areaextraction processing data corresponds to the determined lesion areaextraction processing data; and issuing a warning if the judgment madein the judgment step is not corresponding.
 11. The lesion areaextraction method of claim 8, wherein: the method further comprises thestep of recording a second radiology report generated, at a differenttime, for the same patient as the one of a first radiology report whichis the radiology report; when the character string is a character stringrelated to a second character string, having a lesion descriptioncharacter, provided in the second radiology report, the step ofdetermining lesion area extraction processing data is a step in whichthe lesion area extraction processing data is determined based on thelesion description character of the second character string.
 12. Thelesion area extraction method of claim 8, further comprising the stepsof: recording a second radiology report generated, at a different time,for the same patient as the one of a first radiology report which is theradiology report; when a character describing a lesion at a positioncorresponding to the position of a lesion area is provided in the secondradiology report, obtaining lesion area extraction processing data forthe described lesion; judging whether or not the obtained lesion areaextraction processing data corresponds to the determined lesion areaextraction processing data; and issuing a warning if the judgment madein the judgment step is not corresponding.
 13. A computer readablerecording medium on which is recorded a program for causing a computerto function as a lesion area extraction apparatus for extracting alesion area from a medical image, the program causes the computer tofunction as: an extraction process recording means for recording aplurality of lesion area extraction processing data generated in advanceaccording to a plurality of types of lesion areas; a report recordingmeans for recording a radiology report which includes a character stringhaving a lesion description character and being related to positioninformation of a lesion area in the medical image; an extraction processdetermination means for determining lesion area extraction processingdata used for the extraction from the plurality of lesion areaextraction processing data recorded on the extraction process recordingmeans based on the lesion description character provided in theradiology report; and an area extraction means for performing theextraction using the determined lesion area extraction processing dataand the position information of the lesion area related to the characterstring.
 14. The recording medium of claim 13, wherein: the program is aprogram further causes the computer to function as a display means fordisplaying the radiology report and a specification means for specifyingthe character string in the displayed radiology report; the extractionprocess determination means is a means that performs the determinationbased on the lesion description character of the specified characterstring; and the area extraction means is a means that performs theextraction using the determined lesion area extraction processing dataand the position information of the lesion area related to the specifiedcharacter string.
 15. The recording medium of claim 13, wherein: thereport recording means is a means that further records a secondradiology report generated, at a different time, for the same patient asthe one of a first radiology report which is the radiology report; thecharacter string is a character string also related to a secondcharacter string, having a lesion description character, provided in thesecond radiology report; and when the character string is related to thesecond character string, the extraction process determination means is ameans that determines the lesion area extraction processing data basedon the lesion description character of the second character string. 16.The recording medium of claim 13, wherein: the report recording means isa means that further records a second radiology report generated, at adifferent time, for the same patient as the one of a first radiologyreport which is the radiology report; and the program is a program thatfurther causes the computer to function as: an obtaining means for, whena character describing a lesion at a position corresponding to theposition of a lesion area is provided in the second radiology report,obtaining lesion area extraction processing data for the describedlesion; a judgment means for judging whether or not the lesion areaextraction processing data obtained by the obtaining means correspondsto the lesion area extraction processing data determined by extractionprocess determination means; and a warning means for issuing a warningif the judgment made by the judgment means is not corresponding.
 17. Therecording medium of claim 14, wherein: the report recording means is ameans that further records a second radiology report generated, at adifferent time, for the same patient as the one of a first radiologyreport which is the radiology report; the character string is acharacter string also related to a second character string, having alesion description character, provided in the second radiology report;and when the character string is related to the second character string,the extraction process determination means is a means that determinesthe lesion area extraction processing data based on the lesiondescription character of the second character string.
 18. The recordingmedium of claim 14, wherein: the report recording means is a means thatfurther records a second radiology report generated, at a differenttime, for the same patient as the one of a first radiology report whichis the radiology report; and the program is a program that furthercauses the computer to function as: an obtaining means for, when acharacter describing a lesion at a position corresponding to theposition of a lesion area is provided in the second radiology report,obtaining lesion area extraction processing data for the describedlesion; a judgment means for judging whether or not the lesion areaextraction processing data obtained by the obtaining means correspondsto the lesion area extraction processing data determined by extractionprocess determination means; and a warning means for issuing a warningif the judgment made by the judgment means is not corresponding.